{
 "cells": [
  {
   "cell_type": "code",
   "id": "initial_id",
   "metadata": {
    "collapsed": true,
    "ExecuteTime": {
     "end_time": "2025-01-14T06:55:17.413822Z",
     "start_time": "2025-01-14T06:55:17.395826Z"
    }
   },
   "source": [
    "# 随机梯度下降法\n",
    "import numpy as np"
   ],
   "execution_count": 27,
   "outputs": []
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "$$\n",
    "y = 4 + 3x + noise\n",
    "$$"
   ],
   "id": "a52e68331b76c838"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-01-14T06:55:17.429048Z",
     "start_time": "2025-01-14T06:55:17.424071Z"
    }
   },
   "cell_type": "code",
   "source": [
    "# 创建数据集 X、y\n",
    "X = 2 * np.random.rand(100, 1)\n",
    "y = 4 + 3 * X + np.random.randn(100, 1)\n",
    "X_b = np.c_[np.ones((100, 1)), X]"
   ],
   "id": "eea055088ee03b0e",
   "execution_count": 28,
   "outputs": []
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-01-14T06:55:17.445049Z",
     "start_time": "2025-01-14T06:55:17.432047Z"
    }
   },
   "cell_type": "code",
   "source": "X_b",
   "id": "5175701bc20a34dc",
   "execution_count": 29,
   "outputs": []
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-01-14T06:55:17.461065Z",
     "start_time": "2025-01-14T06:55:17.447050Z"
    }
   },
   "cell_type": "code",
   "source": "y",
   "id": "9ae89f5c521fb6fb",
   "execution_count": 30,
   "outputs": []
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-01-14T06:55:17.476066Z",
     "start_time": "2025-01-14T06:55:17.462049Z"
    }
   },
   "cell_type": "code",
   "source": [
    "# 设置超参数\n",
    "n_epochs = 10000 # 迭代次数\n",
    "m = len(X_b) # 样本数量，也就是每个轮次迭代的批次\n",
    "learning_rate = 0.001 # 学习率"
   ],
   "id": "eefca676e7174231",
   "execution_count": 31,
   "outputs": []
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-01-14T06:55:17.492063Z",
     "start_time": "2025-01-14T06:55:17.478049Z"
    }
   },
   "cell_type": "code",
   "source": [
    "# 初始化参数\n",
    "theta = np.random.randn(2, 1) # 随机初始化参数"
   ],
   "id": "2dc6db24da949dfe",
   "execution_count": 32,
   "outputs": []
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-01-14T06:55:17.508047Z",
     "start_time": "2025-01-14T06:55:17.493049Z"
    }
   },
   "cell_type": "code",
   "source": "theta",
   "id": "e149345f8e49e850",
   "execution_count": 33,
   "outputs": []
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "这里是双层 for 循环，第一层 for 循环是分轮次来计算，第二层 for 循环是分批次来计算\n",
    "`random_index = np.random.randint(m)` 是随机的从数据集中取一条数据的索引，然后根\n",
    "据索引来切片操作取随机出来数据的 $X_i$ 和 $y_i$，每次 W 更新所需要的梯度的求解只用到一条\n",
    "样本的信息。"
   ],
   "id": "9746251619930ad0"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "$$\n",
    "gradient_i= X^T \\cdot (X\\theta-y)\\\\\n",
    "$$"
   ],
   "id": "73ac39e7bd41c79a"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-01-14T06:55:21.671057Z",
     "start_time": "2025-01-14T06:55:17.509066Z"
    }
   },
   "cell_type": "code",
   "source": [
    "for epoch in range(n_epochs):\n",
    "    for i in range(m):\n",
    "        # 求 gradient\n",
    "        random_index = np.random.randint(m)  # 随机的从数据集中取一条数据的索引\n",
    "        xi = X_b[random_index:random_index + 1]  # 取出随机的一条数据\n",
    "        yi = y[random_index:random_index + 1]  # 取出随机的一条数据\n",
    "\n",
    "        gradients = xi.T.dot(xi.dot(theta) - yi)\n",
    "\n",
    "        # 更新参数\n",
    "        theta = theta - learning_rate * gradients\n",
    "\n",
    "theta"
   ],
   "id": "100db53d12bf8d55",
   "execution_count": 34,
   "outputs": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
